2024-09-12
Throughout this class, you’ll periodically be asked to read academic articles on a topic that have empirical research.
Sometimes these will contain methods you’ve never encountered before[^1] so its important to know how to get useful information in these situations
(This is true for academics too. There’s a lot of methods out there, and most of us only learn stuff that’s relevant for our own research or teaching)
Abstract: summarizes the following sections in a few sentences
Introduction: lays out the research question in more detail
This section will usually include a literature review that discusses existing findings
It may include a separate theory section that details how the authors’ approach differs from prior work or the limits of existing knowledge on a topic
Methods: talks about the data collection strategy, measurement of the variables, and statistical models
Findings: shows the results of the statistical tests. Might include multiple robustness checks to anticipate potential critiques of the main result
Discussion/Conclusion: summarizes the findings again, discusses implications, and future work
You’ll find a lot of things in methods sections, but not all of them are equally important.
You’ll find a lot of things in methods sections, but not all of them are equally important.
The outcome being explained (Y)
The explanatory variables (X)
The relationship between X and Y
The measurement, data, and model
The alternative explanations
You’ll find a lot of things in methods sections, but not all of them are equally important.
1. The outcome being explained (Y)
2. The explanatory variables (X)
3. The relationship between X and Y
The measurement, data, and model
The alternative explanations
1-3 are crucial.
4 and 5 are important, but only insomuch as they might lead you to doubt 1-3
We advance social movement and diffusion theories by exploring the role of online activities in the spread of the Occupy Wall Street movement. The results from event history analyses suggest that, after controlling for community characteristics, online activities on Facebook and Twitter are associated with the spread of protests.
From Vasi, Ion Bogdan, and Chan S. Suh. “Online activities, spatial proximity, and the diffusion of the Occupy Wall Street movement in the United States.”Mobilization: An International Quarterly21.2 (2016): 139-154. https://meridian.allenpress.com/mobilization/article/21/2/139/82998/Online-Activities-Spatial-Proximity-and-the
What is the research question, what outcome is being explained? (DV)
We advance social movement and diffusion theories by exploring the role of online activities in the spread of the Occupy Wall Street movement. The results from event history analyses suggest that, after controlling for community characteristics, online activities on Facebook and Twitter are associated with the spread of protests.
What factors are predicting/causing the outcome?
We advance social movement and diffusion theories by exploring the role of online activities in the spread of the Occupy Wall Street movement. The results from event history analyses suggest that, after controlling for community characteristics, online activities on Facebook and Twitter are associated with the spread of protests.
Is the IV having a positive or negative effect? Or maybe curvilinear or conditional?
We advance social movement and diffusion theories by exploring the role of online activities in the spread of the Occupy Wall Street movement. The results from event history analyses suggest that, after controlling for community characteristics, online activities on Facebook and Twitter are associated with the spread of protests.
An explanation for why these things are related should be laid out in the theory section if not the abstract.
Once you’ve got this down, you can start thinking about potential problems.
| Issue/error type | Minor issues | Major issues |
|---|---|---|
| Measurement | Lost forms, instrument failure, data corruption | Lack of construct validity, researcher bias |
| Sampling/uncertainty | Lucky/unlucky draws. Insufficient data. | Differential non-response, attrition, p-hacking |
| Model design problems | Heteroskedasticity, autocorrelation, multicollinearity | Omitted variable bias/spurious correlation |
In general: things that cause random “noise” are very easy to deal with by collecting more data or changing our models, but things that cause bias are more difficult.
Why are there four different models here?? What are linear panel regressions? What is a robust standard error?
Regardless of the model: positive sign means positive relationship, Negative sign means a negative relationship. Asterisks mean that the result is unlikely to be a product of random sampling error (but its still possible!)
the boring truth is that there are four different models here mostly to convince people that the authors tested the same thing four different ways and got similar results.
The article text is far more useful for getting a sense of the findings.
What is the outcome the authors are trying to explain?
The dependent variable here is “democratization” and the primary independent variables are “mobilization for autocracy” and “mobilization for democracy”.
“successful protest has the possibility to expand opposition to the incumbent by revealing new information to others”
“reform as a response to popular disquiet may become the more attractive option for the incumbents [compared to repression]”
“counterelites may become emboldened…if collective action reveals greater disquiet with incumbent rule.”
“surveyed experts and asked them to estimate the size and frequency of mass mobilization”
“Electoral democracy index” (also created from expert surveys)
What other factors might explain the observed relationship here?
Specifically, you are looking for things that might be correlated with both the IV and the DV
(you don’t have to take their word for it, but good papers will usually just tell you what they think the results mean)
Try to identify the research question, DV, IV, and the proposed relationship first. (this should all be clear before you get to the methods section)
Read the abstract closely - a good one will preview the main findings.
Try to get a sense of what parts are common practice as opposed to central to the findings. (if the author doesn’t elaborate on why they chose a particular measure or method or what it means, then it is probably okay to ignore it)
Good papers will try to give a sense of the size of an effect and its practical importance, rather than just showing a bunch of coefficients.